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This investigation, part of a longitudinal study of homemaking teacher 
effectiveness, was designed to explore the usefulness of clinical judgments to predict 
teacher success. Clinical judgment is defined as involving the ability to make so^d 
decisions after gathering and evaluating all the pertinent evidence, weighing possible 
alternatives in terms of past experience or normative probabilities, and arriving at 
problem solutions which reflect basic science orientations. The plan worked to 
determine the reliability of the judge’s estimates and to correlate their estimates and 
the composite success scores. Ten judges, including clinical psychologists, guidance 
counselors, and home economic teacner educators each analyzed 16 randomly 
assigned cases, providing two evaluations per case. Statistical analysis revealed 
significant differences among judges, subjects, and measures, and that correlation of 
judges* estimates and composite success scores was not feasible. All types of 
predictive data were considered useful as they were referred to in the judges 
t evaluation. The appendix contains interpretations of test scores and samples of the 
judges* rating sheet. (FP) 



CLINICAL EVALUATION OF PREDICTIVE DATA 
FOR PROSPECTIVE HOME ECONOMICS TEACHERS 



Ardvce Lucile Gilbert 



u, nmmm or Hwirn wmhm & mmu 

mm O f mmm 



THIS O0CUHIHT HAS m mmm imm as Riaiyse 

HmH OR OROAHIIATiOH OMIHATIHO IT. POIHTS Of VIEW OR ; 
STAT© DO HOT HKESSAWIY RfPKSfHT OfflCIAl OfflCS Of lOOCATfOH 
POSITIOH OR ROliry 



Oltoical Evaluation ol i?rodictlve Data 
Fop Fpospcctive Ho»® Ecoaoiaics Teaoheps 

by 

Apdyco tucllc Gilbept 



Dlpocted by 

Hester Chadderdoai Fh.D, 

FrotessoPf Home EconoBilcs Edueatioa 

Thesis conducted as part o£ Project No. A5f Prediction o£ Success o£ 
Graduates o£ Iowa State Univarsity in Teaching Vocational Home Economics, 
under grant from Iowa Departraeut of Public Instruction, Division of 
Vocational Education, PL88-210 Sec. 4(a). 



Iowa State University 
of Science and Technology 
Ames, Iowa 
I960 










by 

Ardyce Lycila Gilbert 

A Thesis bybmittcd to the 
Graduate Faculty in Partial fulfillment of 
The Eequircmenta for the Degree of 
MASTBE OF &CIEN0E 

Major subjects Home Economics Education 



Approved s 



In' tibax^e' o£'Wjoir'’Wdrlc 






Iowa 6tate University 
Of bcicncc and Technology 
Ames, Iowa 



1966 



TABLE 0? OOaTEKTL 

?ar^e 



imODUOTIOS 1 

.REVIEW 0-? LITEOATUSE 3 

>5BT!iOO OE PAOCELUiffi 11 

FIKDlNGt. AND DlSiC0fc£,'XON 21 

iWajARY 27 

LITEllATl <E CITED 30 

ACKNOJLEDGEZCNTo 32 

< APPENDIX A. INTEtlPlUSTATIONL OP TEbT 3C0RE3 33 

APPENDIX B. JUDGE >i> RATING 3HEET 37 

o 



» 



# 





turn 




I 

7ov many years researchers and educators have been aware 
of the responsibility for and the importance of the decisions 
involved in accepting candidates for a teacher education 
program, iJtudics have been conducted with the hope of es« 
tabXishing a sound basis for judgments but those desij^ed to 
predict the effectiveness of teachers have produced disap» 
pointing results. One possible reason for failure is that the 
statistical analysis of the predictive data is inadequate. In 
a few studies involving prediction clinical judgments were 
used to advantage! hence, it seemed feasible to try this method 
of analysis in predicting teacher success. Some clinical psy- 
chologists believe that the clinical is superior to the sta- 
tistical in predicting behavior because cf the human element 
involved, 

A longitudinal research project^ is currently being con- 
ducted at Iowa otate University to predict effectiveness of 
homemaking teachers. The selection of criteria and predictive 
data was begun in 1958 and, in a recent exploratory study, 
Crabtree <3) used a statistical analysis to investigate the 
relationship of the prediction and the criterion measures. 

The correlations between the two were positive but inadequate 
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for prt,diction of voaeljur oaerooo ovi oa iiadividual baoii 
prooont otudy wao dooij^ned to oKpXora tlio yoefulnoos of clini- 
cal jud, meats to predict teacher oacceso. 

several interpretations of the term "elinical judgments” 
arc found in the literature but !Thorne*s definition has been 
accepted as pertinent to the present study; 



Clinical jad^^ient is operationally defined as in- 
volving the ability to make sound decisions after 
gathering and evaluating all the pertinent evidence , 
wei^^hing possible alternatives xn terms of past ^ 
experience or normative probabilities , and arriving 
at problem solutions which reflect basic science 
orientations (the cultural value system against 
which scientists operate). <14, p. 128) 



SEVXBW OF LXTSHA^UF^ 



The concern of educators for raany years with the quality 
of teachers being educated has led to numerous investigations 
in an effort to identify and measure characteristics of a 
successful teacher# bcveral have been directed voword pre*" 
diction of teacher effcotiveness but none was found that used 
the clinical method of analyzing data collected for prediction. 
The studies here will be limited to those concerned with va- 
lidity and reliability of clinical judgtaents and clinical pre-» 
dictions of performance based largely on non- projective types 
of tests. Projective data differ so greatly in nature from 
those used in the present investigation that research based 
largely on them was eliminated. Investigations involving the 
prediction of teacher effectiveness at the secondary level 
will include only recent investigations to supplement Grab- 
tree’s review. 

The use of clinical judgments in predicting performance 
has been a controversial issue for some years, particularly 
among clinical psychologists and research workers in that 
field. Meehl (10) was among the first to question clinical 
judgments as being too subjective. Although more recently 
(11) he appears to have modified his view, he continues to 
support the actuarial method as the only sound one. The as- 
sertion of several writers, including Meehl, that few cli- 
nicians have demonstrated validity and that they make no 






bottw^r tiimi 

cboXor.i^^ts to rotaliato. Holt 



hao eayoud qU%qc oliidcal poy-^ 
(6) ar*<3 llaoimo (14) baoe tbe'Xr 



objootiono to thooa ar.^uvnoncs on the pm»iioc that the studies 
cited coftvparcd the perfonnance of the beat teats with the 
dictions of unspecified groups of clinicians. Thome points 



out that 

research which sariiplco the averaged judgment of good 
and bad clinicians tends to produce no better than 
chance prediction because the superior judgrocnta of 
the good clinicians arc balanced off by the invalid 
judgunents of the poor. The crucial test of Meehl»s 
hypothesis is to compare the judgments of the best 
clinicians with the best actuarial predictions. (I5f 

p, 116) 

Holtzman (7) suggested that the relevance of cither method can 
be determined only by analysis of the activities involved, and 
that dissention arises because the supporters of cither method 
tend to oversimplify the problem. He implied that both 
methods are valuable in their ovm context, the actuarial for 
data processing and the clinical for interpretations where the 
human clement cannot be eliminated. 

In an attempt to substantiate his belief that clinical 
judgments are valid , Newton (12) divided 50 subjects into five 
equal groups consisting of the socially adequate, non-hospi- 
talieed neurotics, and hospitalised schiaophrenics . A seven- 
point quantitative scale of adjustment was constructed to aid 
in rating the subjects. Each of 10 psychiatrists and 10 psy- 
chologists was asked to evaluate the clinical materials of 15 
subjects and to rate them in terms of over-all adjustment. 



Newton found a high degree of reliability among tne paychia- 
txusts jud0iienta (.91) and the psychologists judgments (.94), 
and a significant relationship (.86) between the judgments of 
both disciplines, 

oomc of the criticisms regarding validity of clinical 
jud£‘;mcnts were directed at the unfamiliarity by the clinicians 
with the criteria involved and the use of rating scales with 
which they had had little eKperienee. Lewinsohn et al. (9) 
investigated the validity of clinical interpretations from a 
battery of psychological tests commonly used in practice. 

Five psychologists gave blind ratings of test protocols for a 
randomly selected sample of 100 psychotic and neurotic hospital 
patients. The ratings were based on a battery of tests, age, 
and sex of patient and recorded on a 23-itcm rating scale 
which the psychologists had previously helped to develop and 
with which they had had experience. Each judge rated 40 
subjects, thus providing two independent judgments for each 
patient. The criteria were parallel ratings based on the 
patient *s hospital chart and on an interview by a psychologist 
who was unaware of the test results. The authors report that 
’Validity coefficients obtained were predominately in the di« 
rection of supporting the validity of the test ratings” but 
that ”thc validity differed with different ax^'eas of patient 
functioni\ig*” 

Instead of using interview data as the criterion measure, 
Bobbitt and Ne^^nan (2) employed ratings based, in part, on 



6 



f 






interviav/s a« onQ of throe* baooo fo*r predictin'^ the* cucccsa of 
officer candidates in the United J^tateo Coast Guard Academy. 
Each officer candidate v/as interviewed by a psychologist and a 



psychiatrist from the medical departroent of the Academy, who 
were provided with the Personal Data Questionnaire and availa-* 
ble test scores for the candidate. The interviewer assigned 
an overall rating based on a written siammary of his evaluation 
and interpretation of the test scores for each candidate. A 
second prediction measure was the combined scores for three of 
the tests which had been available to the interviewer: 1) 

quantitative ability, 2) verbal ability, and 3) bi-dimensional 
spatial perception. The third basis for prediction was the 
combined scores of the interviewers rating and the tests just 
enumerated. The criterion to be predicted was the degree of 

f 

success during training in the Reserve Training School, baaed 
on academic achievement and adaptability records which were 
considered in the final class standing. These were used to 
classify the candidates into four groups. Bobbitt and Net-nnan 
reported that there wac a direct relationship between the pre- 
diction measures and the success of the officer candidates, 
but that the combination* of the interview ratings and the test 
scores produced a better* prediction than either did separately. 

In a prediction of performance of aviation cadets, 
Holtzman and Sells (S) investigated the possibilities of de- 
* vcloping a hypothesis for the quantitative scoring of a group 
of screening tests. A group of 100 cadets, 50 who had made 
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AdjufcJtmunte to £Uf,ht training, and 50 who hod hean 
UR6uccct>t^£uX 3*Ti coiHpXotiti^^ thc pi?o^5£*aui| wo^o ipandoinXy aoXoctod 
and an oxpcrimcntaX dcai£^ was dcveXoped wherein each of X9 



payehoXogicts rated 20 cadets by two mcthodfes judgment based 
on one test at a time, and judpients based on a gXobaX evaXu- 



at ion of the predictive data avaiXabXe for each subject. Xn 
addition, each judge was asked to state the cues which infXu- 
enced his evaXuation and to indicate on a three«*point scaXe 
the degrees of confidence he had in the vaXidity of his 
judgments. LittXe reXationship between the cXinicaX evaXu- 
ation of the cadets and the measure of fXying success was 



found. However, the amount of agreement among the judges for 
the gXobaX approach tended to be significantXy better than 



chance • 

In three recent studies attempts have been made to predict 
teacher competence. Xn search of college records which might 
be used to predict success FrcchiXX (4) investigated the re- 
lationship of coXXege recommendations and field evaXuationc. 
The coXXege recommendations were based on the records which 
ineXuded; X) entrance test data, 2) academic records, and 3) 
a report on sociaX and community Xife which was imtcd on a 
XO-point ceaXe indicating* the student's degree of strength or 
weakness as a teacher candidate. The entrance test data in- 
eXuded scores from an academic aptitude anamination, three 
EngXish test scores, and XO scores from instruments deveXoped 
in the American GouneiX on Education Study of EvaXuation. The 
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Ihy i^facia x^oint uvo^mrs- c. 08 iipwtQ<J £or aij;ht 



yul^jia t-iiiattC'f «rwux->iti‘*a, aiiU tlse Wi’/ a lativc^ to i^Qiial fmd 
contmunity' life wae imeecJ laviplf on r&tini^o umda by faculty 



an<3 sux^ervicore of etudent teaching* Sie field evaluations 
were obtained near the end of the first year of teaching and 



again near the end of the fifth* frincipala, superintendents , 
and supervisors or vice principals rated the teacher on the 
basis of professional and personal qualities at these two 
periods. Freehill reported that there was a positive re- 
lationship between the principalis judgment and the college 
rccojhmendatione, but that on professional qualities the later 
field evaluation agreed mo^e with the college evaluation than 
did the earlier field evaluation* 

Sprinthall, et al. (13) ignored the *'static personal 
traits which cannot be made operational** and concentrated on a 
conceptual framework for prediction and evaluation of teaching 
success based on observable teacher classroom behavior and its 
relationship to cognitive flassibility-rigidity scores of psy- 
chological tests* Tt^cnty-eight subjects were randomly se- 
lected from a population of (graduate students enrolled in a 
master of arts teaching program which involved one year of 
study. During the summer, seven weeks were devoted to in- 
tensive supervised student teaching, while half of the follow- 
ing academic year was concentrated on full-time classwork and 
the other half on intern supervised teaching in a local school 
system* The predictive data were obtained from two 
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psychological tects, Roi^schach and Visual Impression lest 
(VXT) , which were administered before the subjects beg^n 
student teaching, Ihe criterion measure consisted of ratings 
of teacher behavior based on observations during a 60-minute 
sample of the student teaching period, and subsequent super- 
visory-planning conferences, Tho. results indicated that ef- 
fective teaching and cognitive fleKibility-rigidity are 
related, bprinthall ct al, , however, suggested that the pre- 
dictive data need to be refined and that complete follow-up 
information of success as a full-time teacher be made to vali- 
date the criterion measure, 

Xn a study of homemaking teachers Crabtree (3) analyzed 
data that had been collected to determine the relationships 
between selected predictors and success criteria using subjects 
who were graduates of Xowa State diversity. During their en- 
rollment in the University predictive data were collected by a 
battery of instruments; The Guilford -Timmerman Temperament 
Survey (GZTS) and the Minnesota Counseling Inventory (MOX) to 
measure personality traits j the Just Suppose Inventory (JSX) 
to indicate certain attitudes j and the Johnson Home Economics 
Interest Inventory (JHEXI) to obtain an estimate of vocational 
interests. Also included was the cumulative quality point 
average (CQPA) at the end of the sophomore year, 2^ae three 
criteria used to determine the effectiveness of a teacher were 
teacher-pupil rapport, pupil gain in ability to apply general- 
izations in solving problems in home economics, and the 
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adjus^traant of the teacher to school and community* Sixty-four 
homemaUin^ tcachera were ii\cXuded in her anaXyais. A pancX of 
judj:,06 rated the predictors and criteria in tcri^ns of their 
reXative importance and an adaptation of the J-coefficiont pro- 
cedure was used to provide weights for each* X’he weighted 
predictors were cuRmied to obtain composite prediction scores 
and, *»iroiXarly, composite criteria scores were secured* When 
'^hesc were intercorreXated she found that academic achievement 
had a significant but Xow correXation with the composite cri- 
terion score* AXso scores on attitudes toward Xow- income 
groups and toward middXe and upper eXass groups correXated 
posit iveXy with the composite criterion and aXl individuaX 
criterion scores. AXthough severaX scores from the predictive 
data exhibited positive correXations with the composite success 
scores, the composite scores were not hi(^ enough to use in the 
prediction of teaching success for an individuaX* 
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KETIIOD OF X^ROOEDUm 
Purpose of ^tudy 



This study is part of a lougitudinal study to predict the 
effectiveness of horocinaking teachers who are graduates of 2owa 
state University, Since the statistical approach used by 
Crabtree did not produce satisfactory estimates for the pre- 
diction of the success of an individual student, a clinical 
analysis was explored using the predictive data available for 
80 first -year teachers. The present study was designed also 
to determine the number and the type of judges needed to make 
reliable estimates. 

Description of Population 

An attempt was made to obtain data for all graduates who 
taught in Iowa during the period 1961-1062 to 1965-1966. Be- 
cause the achievement tests used to measure pupil gain were 
based upon the Iowa Homemaking Curriculum Guides, only those 
. graduates who taught in Iowa schools were included, bince 
most first -year graduates teach classes at the ninth- or tenth- 
grade level, it was not feasible to develop instruments to 
measure success of the few who taught only at other levels. 
i?or this reason the population is further limited to those 
teaching Komeraaking I and II classes in Iowa. 

Table 1 presents information concerning the number of 
home economics education graduates of Iowa utatc University 



jfroffl 1963. to 1966 who X nvk6/ oi^ XX elaooec In 

Iowa, ana the raaaena for eKoluding a portion of the popu- 
lation from thifi atudy. 



^Pable 1 



Graduates of Iowa btatc University who taught Home - 
making I and/or IX classes from 1961-1962 to 1965- 
1966 



Graduates 


1961- 

*62 


1962«* 

'63 


1963- 

'64 


1964- 

•65 


1965- 

•66 


TOTAL 


Included in study 


3 


17 


IS 


29 


16 


80 


Incomplete predictive 
data 


17 










17 


Refusal to cooperate'^ 


2 




3 


1 


2 


8 


Resignation before end 
of year 


1 








1 


2 


Incemplcte success 
data*^ 




6 


6 






12 


Errors in adminis- 
tering test 






1 






1 


Late placement of 
teacher 




1 






1 


2 


TOTAL 


23 


24 


25 


30 


20 


122 



•^Superintendent or teacher 
*/f*A‘Largely incomplete administrator's ratings 



Incomplete data were available for a large number of the 1961- 
62 graduates because the second personality inventory was 
selected for use too late to be administered to most of these 
students. 



Of the 30 teachers for whom complete data were available 
54 taught both Homemaklag I and XX classes, 21 taught Home** 
making X but not Homemaking XX classes, and 5 taught only 
Homemaking XX classes, 

Predictive Data 

These data included the cumulative quality point average 
and a battery of four instruments. The Guilford -2'iitmnerrnan 
Temperment Survey (GZTs) and the Minnesota Counseling Inven- 
tory (MCI) were employed to obtain an estimate of personality 
traits, and the Johnson Home Economics Interest Inventory 
(JKEII) to determine vocational interests. The Just Suppose 
Inventory (JaX), which is not yet published, involves atti- 
tudes toward other persons and groups,^ The student is asked 
to project herself into each of the 12 situations which might 
be encountered by a teacher and to select statements which re- 
veal her attitudes. The situations relate to: acceptance of 

changing conditions in our society, especially broken homes 
and mothers working and of parents with little or much edu- 
cation} adaptability to communities of different sizes and to 
various areas within a community, i,e. industrial sections of 
a city, slum distz'icts, suburban areas} tolerance of 



^Copies arc on file in Department of Home Economics Edu- 
cation, Iowa 5tate University. Permission was obtained from 
Ruth Lehman, Ohio 5tate University, to use this inventory. 
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£orcx.’n-bom and ethnic (^roups other than one* a ovmi 
for different relifioni? and for families in the Xov7, 



respect 
middle , 



and upper income groups | understanding problems involved when 
working with low X.Q# or doliquent students, persons living in 
a three-generation family home 5 and attitudes toward parents 
in relation to concern about their children's welfare. 

The students who entered Iowa otate University as 



freshmen were administered two of the inventories either at 
the end of the freshmen year or the beginning of the sophomore 
year and the other two at or near the end tf their sophomore 
year, i&tudents who transferred into the University or into 
the Home Economics Education Department reacted to these 
inventories soon after the transfer. For the students 
entering as freshmen the cumulative quality point average was 
recorded when they were formally admitted to the* home eco- 
nomics teacher program, commonly at the end of their sophomore 

* 

year. For the transfer students this average was recorded 
when they applied which usually was after the completion of 
two quarters of work at Iowa State University. 

These data were supplemented in the present study with 
information concerning pre-college work experiences and ac- 
tivities, a statement by the advisor of the student *s 
strengths and limitations, and the student’s statement of 
motivation to teach. These were obtained from the Application 
for Admission to Teacher Education Curriculum in Home Eco- 
nomics. Also included were statements obtained from the 
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Euporti** which iudiccwc chu**ftctc<r3.£#t'4.Cfc# be** 
havxoc* obMcrvaxI by the clac^room teachex’^ in the CcXlcr.^ 

Horn SconoiAic&;. It waa hoped that the work e::perlcncea and 
the activities v/ould give oome indication oZ leadership quali-» 



tieSf that the statement e of the advisors and the teachers 
would supplement information concerning the ability to relate 
to other persons and that the student »s statement would pro- 
vide further insights into the personality and abilities of 



the prospective teacher# 



Prediction of Success 



X'en judges analyzed the predictive data and made esti- 
mates for the 80 subjects. Each judge was provided the infor- 
mation previously described and also some data to aid in 
interpretations # 

For the GZTS a summary^ was developed from the Manual of 
Instructions and Interpretations <5) of the qualities which 
describe a high and a low scorer for each of the 10 personali- 
ty characteristics. In addition the scores of each subject 
were recorded on the profile based on the responses of 389 
college women reported in the manual. These data were sux^plc- 
mented by means and standard deviations derived from a sample 
of 100 sophomores in the Department of Home Economics 



copy of Interpretation of Scores - Esetremes may be 
found in Appendix A, 

j 



n /• 

io 



Education, A clear plastic 



overlay containinf the ir.%'>an and 



the one standard deviations from the meaii was supplied each 
judge. 

Descriptions of the high and the low scorers for the 



personality traits roeasured by the MCI were duplicated frora 
the manual (1), I’he scores of the subjects wore recorded on a 
profile based on the response of 367 sophomore, junior, and 
senior students who were majors in home economics education. 

The profile for the JHEIX has been published^ but since 
it is based on scores of freshmen home economics students, a 
clear plastic overlay indicating the means and the ^ one 
standard deviation from the means based on a sample of 100 
sophomores enrol.led in home economics education was provided 
to facilitate analysis of the scores on this Inventory. 

No manual is available for the JbX, therefore an expla- 
nation of the attitudes which might be expressed by high and 
low scorers was made from the statements included in the 
Inventory.^ A profile which had previously been developed, 
based on the response of 330 sophomores enrolled in home eco- 
nomics education, was used to record the score of each subject. 

Of the 10 judges selected, two were clinical psycholo- 
gists, five were guidance counselors, and three were staff 



^^'Published by The Iowa btate College Press, Press 
Building, Ames, Xowa. 

^A copy of the Just Suppose Inventory: Interpretation of 

Scores may be found in Appendix A. 
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mambere of the Horae Bconomxco Education Department. 



Each 



judese analysed 16 eases which had been randomly aeeignedj this 
provided two estimates for each case. An eleven^point scale 



was used to determine the degree of certainty of the esti- 
tioxuB*^ tChe judges were directed to base their estimates on 



r** r' 
HI4.* 



the likelihood of the student being successful in a one-teacher 
departs'.ent in a high school with an enrollment of less than 
400 students, located in a relatively small Iowa town, a popu- 
lation of 1200-7000, with few lower-class families, This de- 
scription was based on information concerning the most frequent 
teaching situations of the teachers involved in the study. It 
was assumed that the majority of horae economics education 
graduates of Iowa State University would be employed in simi- 
lar situations. Because a few of the graduates taught in 
urban coiTimunities, a second estimate was also made relative to 
success in a larger urban school. In addition, the judges were 
asked to explain any score below 6, with the hope that the 
explanation would be useful in understanding differences among 
judges and in determining which data to continue to collect. 



Success Data 

The three criteria used in the project for determining 



6a copy of the Judge Rating bhcet may be found in 
Appendix B. 
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teachesT effactivonesc werei pupil gain in ability to apply 

generalisations, pupil-teacher rapport, and teacher adjustment 
to school and community# 

Pupxl gaxn was determined by two £o£^s o«t two achievement 
tests. Form A, administered at the beginning of the school 
year, and Form B, administered near the end of the school year 
to Horacmaking I and IX classes, The tests were developed by 
the project leader to assess the gain in ability to apply 
generalizations in solving problems in home economics. The 
Homemaking 1 test included five areas of homemaking; food and 
nutrition, teKtiles and clothing, child development, family 
relations, and housing. The Homcmaking II tests included 
these with the CKception of child development since the state 
curriculum guide did not contain a unit in this area at this 
level, A class mean for Horacmaking I classes was computed for 
each teacher by subtracting the sum of the scores on Form A 
from the suras on Form B, and dividing by the number of pupils 
who completed both forms | the same procedure was used to obtain 
a mean for Horacmaking II c lasses, 

Fupil-teacher rapport data were collected by adminis- 
tcrin?i two £o«fls of the &ETC inventory, one for Homemaking I 
and one for Homeiaakina II classes. Those inventories consist 
of ctateiuents about the homemaking teacher and class to which 
the pupil indicates his feelings by agreeing or disagreeing. 

The items relate to the teacher's interest in, undorstandin" 
of, and attitudes toward the pupil, her willingness to help. 



avid the auiouTit or kind holx> ^ivon to the pupil* kuch 
fuvorut»lc* rc4i»pon£*o v/ut# ^^xvgh u vnluo oM ono 5 nil otliox* re* 



u^QUQQG *x value of sero# Claaa tacans for each teacher were 
computed by summing the scores for the classes in Homemaking X 
and in Homemaking XX and dividing each by the total number of 



pupils# 

An estimate of teacher adjustment to school and community 
was made by the school administrator and* recorded on a special 
form designed for this purpose. The factor analysis made by 
Crabtree yielded two single**item factors and two clusters of 
items* The former items were Physical Health of Teacher and 
Judgment in Discuscion of Personal and Professional' Problems. 
The two clusters involved Management of Department and Re** 
lations with school Personnel, Pupils, and Community* Numeri- 
cal values of 1-6 were assigned to the responses and numerical 
values of the clusters were obtained by summing the scores for 
the responses to the items contained in the cluster* 



Treatment of Data 



In the present investigation the estimates of 10 judges 
were tabulated and the data were analysed to obtain the degree 
of variance due to judges, subjects and measures. 

A reliability coefficient was computed for one judge 
using the following formula; 



20 



« 



# 



a 



Where 



K 



3 






/7** H* 

nN 



” reliability of one judge 

(7*1 ss error variance 
2 

IP^ » subject variance 



n » number of judges 
N s= number of estimates for each judge 
An estimate of the reliability coefficient for ten judges 
employed the formula; 



1 + <n - 1) Rj 



Ij « reliability of 10 judges 
Rj = reliability of one judge 
n = number of judges 



t 
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A'ANlSWOis /a© DltCUi-ilOK 

To the usei^uXiriecs of clirdcal judgrfwittij to 

predict teacher effectiveness , two analyses of data were 
planned? 1) to determine the reliability of judges* estimates 
and 2) to correlate their estimates and the composite success 
scores. 

An analysis was made to determine wherein there was vari- I 

ance due to judge t subject, measui:^, and the interaction be- 

; 

tween them# The measures are the two est:bnates each judge ! 

assi^ed the subject as he evaluated her effectiveness as a I 

homcmaking teacher* The results are presented in Tabic 2« I 

The F values were calculated using the error mean square as I 

the denominator, with the except ion of M, for which the vari— I 

ance of the interaction of CM was used# The analysis yielded ' 

F values that arc highly significant for judges, subjects, and 
measures* 

Since one of the purposes of the present study was to 

' 

explore the accuracy of clinical judgments, reliabilities of 

judges were computed. A reliability coefficient of .142 was I 



obtained, thus theoretically limiting the validity coefficient 
of the judgments to #37 • Because the estimates of ten judges 
were employed in the prcs'*nt investigation, a reliability coef- 
ficient for the ten was estimated. The result was a relia- 
bility coefficient of .623, which could not yield a validity 
coefficient above .79. This finding would indicate that even 
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Table 2 Analysis of variance of judf^es, subjects, and measures 



feourcc of 
variation 


bum of 
squares 


d 1 


$ 

Mean 

square 


F value 


Subjects (C) 


720.620 


79 


9.122 


4.30* 


Measures (M) 


20.503 


1 


20.503 


23,06* 


C M 


70.2^^6 


79 


.889 


.42 


Judges <J) 


16S.6S3 


9 


18.406 


8.68* 


J M 


6.903 


9 


.767 


.36 


Error 


300,917 


1^2 


2.119 




Total 


1284. Qt(2 









*Signif leant beyond .001 level 



thoujih the ten judges evaluated each caae and there was no 
error variance in the criterion measure e, the validity of pre- 
diction of success for an individual would be inadequate. In 
order to increase the reliability and the validity coefficients 
of the judges* estimates, at least twice as many cases and/or 
judges need to be included. This, however, is not practical or 
feasible because of the time involved for evaluating each case. 
In view of the evidence revealed in the analysis of variance 
and the reliability of the judgments it was decided not to 
complete an analysis of the judges* estimates and the composite 
success scores of the subjects. 

bitico the reliability of the judges was low, the data 
were examined to discover possible explanations. The judges 
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23 



had been directed to j^lve two eetimte£? of eueceoe for each 
oubjcct, one for teachings in a ©mall eoauf»unity and the other 
in a larger urban area. Both eotimates were deoii^nated by a 
numerical vatini^ on the eleven-point certainty acale. l*hc 
distribution of the judges* first estimate is presented in 
Table 3. Xt reveals a skewed distributionj the judges tended 



Table 3 Number of subjects placed at each degree on the 
certainty scale by judges 



Judges 








Desirees of certainty 








b 


"i" ” 


2 


3 


h 


5 


6 


7 


3 




""lb 


lAt 










3 


2 


1 


5 


4 


1 




B 






2 






2 


2 


2 


3 


3 


2 


C 






2 




1 


2 


3 


2 


4 


2 




D 








2 


4 


3 


6 


1 








E 








1 


2 


1 


3 


2 


6 


1 




F 






1 


1 


1 






2 


4 


2 


5 


G 










2 


3 


2 


1 


6 


1 


1 


H 














3 


4 


5 


3 


1 


1 












2 


4 


7 


3 






J 












1 




6 


6 


3 




Total 


0 


0 


5 


4 


13 


16 


18 


37 


42 


16 


9 



to rate more subjects tov/ard the upper end of the continuum. 
This was to be expected, however, since the sample included 
only those students who had been screened before admission to 
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the teacher education program. The estimates o£ three judges 
were markedly different. Judges H and J tended more than the 
others to use the upper end of the continuum and Judge D the 
(nxdldl Xg ^ 

An analysis of the first estimates of the pairs of judges 
for each case revealed that all judges differed at least three 
points on one or more cases. One possible explanation for 
these differences might be the lack of the judges* experiences 
with the certainty scale. 



Table 4 The number of point differences between first esti- 
mates of judges 



Judges 


Point 


differences 


between .iud'xcs first estimates 


0 


1 


2 


3 


4 


5 


6 


7 S 9 10 


A 


5 


4 


4 


3 










B 


5 


6 


3 






2 






C 


4 


5 


2 








2 




D 


4 


3 


1 


4 


2 






2 


E 


3 


4 


4 


2 


2 




1 




F 


2 


3 


5 


2 


1 


« 


1 


2 


G 


3 


4 


8 


1 










H 


6 


4 


3 


1 


1 




1 




I 


6 


5 


4 




1 








J 


4 


4 


1 


3 


3 


1 
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The data in Table 5 indicate that Judgea 0, jp, and J exhibited 
tnis point diTfarence more than any other judge on both the 
first and second estimate , It is suggested that the cases 
estimated by judges D, F, and J be re-evaluated by the judges 

who less frequently disagreed and another hnalysis. be made to 
determine reliability. 

When a judge assigned an estimate of five or less, he was 
asUed to indicate the reasons for the decision. Upon ex- 
amining these , it was found the judges who agreed on the esti- 
mates for a case tended to select similar bases. The reasons 



Table 5 



?href ofmor^pcints’^''®® disagreed by 




2S 



by bhc juJf'tjti inuOc irciJcronce. to tilX o£ the types o£ Oete 
fJvcn them; hence, it uppeare that all «re uucCul in pre- 
diction, In one case a jndge sugKoeted the need £or addition- 
al infomtation, i.c. age o£ subject and size of coniraunity in 
which she had lived. 
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Tbifi investigation is part of a longitudinal research 
project being conducted to predict the effectiveness of home- 
making teachers v/ho are graduates of Iowa btate University, 
The purpose of the present study is to CKplore the usefulness 
of clinical evaluations for prediction since a recent sta- 
tistical analysis of the data revealed the prediction formula 
inadequate for reliable estimates of an individual. The plan 
was to determine the reliability of the judges* estimates and 
to correlate their estimates and the composite success scores. 

Predictive data collected in the longitudinal study, 
which were available *f or 80 first-year homemaking teachers,* 
included the cumulative quality point average and a battery of 
four instrumants: the Guilford-Zimmorraan Temperament Survey 

and the Minnesota Counseling Inventory to measure personality 
^he CTohnson Home Economics Interest Inventory to indi» 
cate vocational interests, and the Just Suppose Inventory to 
determine attitudes toward other persons and groups. These 
data were supplemented with information concerning pro-college 
work experience and activities, an estimate by the advisor of 
the student's strengths and limitations, and the student's 
statement of motivation to teach. 

Each of ten judges, including clinical psychologists, 
guidance counselors, and members of the Home Economics Edu- 
cation Department, analyzed 16 randomly assigned cases, thus 
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providinf; two evaluations for each case* An eleven-point 
scale was used to determine the degree of certainty of the 
estimation* The were ashed to evaluate i*he student 

twice, as a teacher in a small community and also in a larger 
ur*ban area. In addition they were to indicate reasons for a 
score less than 6, with the hope that the explanation would be 
useful in understanding differences among judges and in de- 
termining which data to continue to collect. 

An analysis of variance yielded statistically significant 
differences among judges, subjec1:s, and measures beyond the 
.001 level. The reliability coefficients computed for one 
judge, .142, and estimated for ten judges, .623, indicate that 
a correlation of the judges* estimates and the composite 
success scores, as previously planned, was not feasible. 

Further examination of the data revealed that the judges* 
estimates tended to be placed near the upper end of the 
certainty scale , whxch was not surprising due to the screening 
process for admission to the teacher-education program. Lack 
of expei^ience with the use of the certainty scale may have 
influenced the judges to make estimates that differed three or 
more points on a case, however, three judges exhibited this 
point difference more than other judges. It was suggested 
that the cases estimated by them be re-evaluated by the judges 
who ICwS frequently disagreed and another analysis be made to 
determine reliability. 

It appears that all of the predictive data are useful 



o 
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hincQ wa6 uiadw to all typoa in th(j rea4»ona |;fivv‘n by 



the jw4d«ea. 



If 



30 



LITiSUATUliE CITEii) 



X. Jsstniio, H- 1'** Layton, V/« L» Miimoaota CounacXiny, 
Invvmtory KatiuaX. New York, Now York, The i^aychoXogical 
Corpo cat ion • X9 37 , 

2* Bobbitt, J« K. and Newman, b. H. PayohoXoyicaX activi** 
ties at the United states Coast Guard Academy. I’sy- 
choXo^icaX BuXXetin 4 Xj 568-579. 1963. 

3. Crabtree, BeverXy. X^redicting and determining effective- 
ness of horoemaking teachers. UnpubXished Ph.0. thesis. 
Arnes, Iowa, Library, Iowa iState University of Science and 
Technology. 1965. 

4. FrcchiXX, M. F. The prediction of teacher competence. , j 

Joui’naX of Experimental Education 31:307-3X1. 1963. ^ 

5. Guilford, J. I^. and 55imr.ierman, Vi^aync b. The Guilford - 
Zimmerman Temperament burvey, Manual of instructions and 
interpretations. Beverly Hills, California, Sheridan 
supply Company. 1949. 

6. Holt, R. R. Clinical and statistical prediction, A 

reforrauXation and some data. Journal of Abnormal bocial 
Psychology 56:1-12. 1958, 

7. Kolteraan, V/. H. Can the computer supplant the clinician? 

Journal of Clinical Psychology 16:119-122. 1960. 

8. Koltsman, U. H. and bells, 8. B. Prediction of flying 

success by clinical analysis of test protocols. Journal 
of Abnormal and bocial Psychology 49:485-490. 1954. 

9. Lewincohn, P. M., Nichols, R, 0., Pulos, L. , Loraont, J. 

F., Nickel, H. J., and bickind, G. The reliability and 
validity of quantified judgments from psychological 
tests. Journal of Clinical Psychology 19:64-73. 1963. 

10. Mcehl, P. E. Clinical vc, statistical prediction. 
Minneapolis, Minnesota, University of Minnesota Press. 
1954. 

11. Meehl, P. E. The cognitive activity of the clinician. 

American Psychologist 15:19-27. 1960. 

# 

12. Newton, R. L. The clinician as judge; total Rorschach 

and clinical case material. Journal of Consulting Psy- 
chology 18:248-250. 1954. 






SL* 






i.zm, escis*. 




31 






13# A,, Wbitely, J# >J# f and Hoahe%*, R# L* A 

study of teacher effect iveiiess. Journal of Teacher Edu- 
cation I7s93-106, 1966* 

14. Thome, F. 0. Clinical Jud^nent; a clinician viewpoint. 

Journal of Clinical Psychology 16:128-134. I960. 

15. Thorne, F. C. Editorial coiument. Journal of Clinical 

Psychology 16:115. I960. 



32 



The writer* to e%pirac»o eincere appreciation to Dr. 

Heotor Ghodderdon for her f,ul<3a«cef cneouraf„ement , and 
patience throntUio\it the entire ctudy. 

Appreciation ia alao e%pres6ed to the /ij^ricuXture and 
Houte Economics Experiment Citation for the opportunity to aerve 
aa a graduate aaai&tant and to participate in the research of 

Project 1413. 

Gratitude is extended to the following persons, whose 
help facilitated the completion of the study: Dr. Leroy 

Wolins for his assistance in the analysis of data; Dr. llusecll 
Canute for his help in obtaining judges for the study; all the 
faculty who served as judges of data; and the administrators, 
teachers, and pupils who cooperated in the collection of data. 



o 



33 



APPENDIX A, IWmmmTAnQl'^i. OF bOOMh 



GUILFQXP XIXKEmN ^SKPERA^IEN^g >%miW.Y 
INTEilPRETATION OF hOO^b - EXTmiEb 



- GSNEIJAL AC0?IVITY 

Hif,h ScovQ - utvow, drive, enerjfy, activity, vitality, 

speed , couraj^e , erithusiafim 

Low Lcore - deliberate, inefficient, inactive, slow 

- REaTFAINT 

High &core - deliberate, consistent, ocif-control 

restraint, seriousness 

Lov/ bcore - impulsiveness, happy-go-lucky, loves ex- 
citement 

- A^GENfANCE 

Kxgh bcore — social coldness, self-defense, leader, being 

conspicuous , bluffing 

Low £»core - social submissiveness , follower 

- SOCIABILITY 

High ccore - many friends, conversationalist, social 

life, likes limeli^t, high social interest 

Low t)Core - few friends, shy, avoid social contacts, 

seclusivcness 

- EK 9 TIONAL STABILITY 

High ocore - even mood, optimistic, cheerful, composed 

Low Lcore - moody, gloomy, pesimistic, daydreams, 

excitable, guilt and worry feelings 

- OBJECTIVITY 

High Ecore - thick skinned, less egoism 

Low bcore - self-centered, suspicious, subjective, 

hypersensitive 

- FRIENDLlNEbO 

High bcore - lack of fighting tendencioo, pacifism, real- 
istic way of treating frustrations, urge to 
please others, desire to be liked, tolerant 
of hostile action, accepts domination, 
respects others 

Low hcore - hostility, fighting attitude, belligerent, 

resentful, wants to dominate, contempt for 
others 






M iMii. 4^43;; 






as 



3/f 



T - TKOUGIOTULNEbfe 

High bcovQ - ob&erving bahavior of others, interest in 

thinlfinf, philosophizing, mental poise, 
reflectiveness 

Low ocore - thoughtlessness, cKtraversion , likes overt 

acts, dislikes reflection 

P - PSafi^ONAL RELATIOHb 

High ;i»core - tolerance and understanding of other people, 

faith in social institutions, good personal 
x*elationc, cooperative 

Low t.core - fault finding, critical of other people and 

of institutions , hypercritical , suspicious , 
self-pity 



M - MAbCULINll'y 

High bcore - not easily disgusted, not fearful, interest 

in masculine activities, hard-boiled, re- 
sistant to fear 

Low bcore - sympathetic, romatic, feminine activities, 

easily disgusted, fearful, etnc^ional 
expreesivenese 

-^hPPose Inventory ; Interpretation of scores 



X* Attitude toward parents ; 



High 

Low 



Parents do their best to understand children and 
do what is best for them? they appreciate the 
efforts of the school. 

Parents arc too «enerous and too permissive with 
thexr children; they are unfairly critical of the 

^Cfi^ooX # 



XX. Attitude toward different size communities ; 

High - People are basically the same, regardless of the 
size of the corni^iunity in which they live. 

Low - Farm families arc behind the times and crude. 

omall towns are dull. City people arc unfriendly 
and critical of others. .‘.v.iiui.y 



III. 




High - Acceptance of divorce as part of todays society. 

Recognition that parent-child relationships can 
T be satisfactory if the mother works. 

Low - Children from broken homes are usually doLincuent 
A mother »s place is in the homo. Divorced 
parents show little concern for their children. 
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IV. Attitiadc toward 

liifh - Generally, families with xoreij^n born parents can 
make valuable contributions to our society. 

Low « ForeigT^ers tend to increase the crime rate and 
lower the standard of living. 

V. Attitude tov/ ard persons with hif^.h or low cdxicational 

Wckgroii^^^ 

High - People are basically the same x'egardlecs of their 
educational level. 

Low - Uneducated people are uninterested in the better 
things of life and do not cooperate with the 
school. Professional people are unwilling to 
accept those who work with their hands. 

VI. Attitude toward lov7*» income groups i 

High « People in slum areas arc victims of circumstance » 
they could do better if given a chance. It would 
be a challenge to try to help them. 

Low - People in slum areas are lazy, indifferent and 
have lov7 intelligence. 



VII. Attitude toward different religions ; 

High - Heligiouc beliefs are personal and differences 

should not influence ones acceptance of a person. 

Low « It would be difficult to v;ork with people whose 
religious beliefs differ from mine. 

VIII. Attitude toward middle and upper»-class ftroups ; 

High - It is the individual in the group that is im- 
portant, not the class they’re from. 

Low - People from the upper-class load an artificial 
life; lack interest in the school and "real 
family life”. Middle-class families are too con- 
cerned about keeping up with the crowd; have 
little control over their children. 



IX. 



Attitudf* toward 
ancf^^elL'i nauent 



teaching'** in a sch ool with many 
student s and ' e re ste d paren 



low 



4L # 

irb « 



I. Q. 



High - These students need encouragement and guidance; 

teaching would be a challenge. 

Low - These students wouldn’t behave or learn anything. 
Their parents are failures. 
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X, 



families o£ tha iaborin ^ c\\(Aai*% 



Ki::>h « Itoay avQ ^cooci-hcarceU , dovm to earth people who 
appreciate what the ochoolc arc doinj^. 

Low « Laborin'!: class farriilics are dull. It would bo 
undesirable to live near them. 



XI. Attitude toward an ethnic ?troup other than ones own ; 

Hi'fn, « The di.f Terences are not really important between 
my j/roup and theirs. 

Low - They have too many objectionable traits; parents 
do not care what their children do. 

XII. Attitude toward a 3 r’cncration family in a home ; 

Hif$h - Harmonious relationships can be achieved if 
family racinbers respect each other. 

Low - Old people are bossy and teen-agers are incon- 
siderate and noisy. 
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APPENDIX B. J0D03»6 RATING SHEET 



K «sMt m .. 
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JUDGE* 3 lUxTXm umT 



CUuC No* 



% 



^vdTudtb¥ 



Direct io:is: 

Acsuinc that this etudent will teach horae econotaico in the 
following eituation; 

a relatively small Xov 7 a town with a population of 1 , 200 - 
7,000 in a community with few lower-class families 
high school enrollment of 100-400 students 
one teacher home economics department 



Considering the evidence presented, what is your estimate of 
the chances that this student would become an effective high 
school teacher of horae economics classes? 

Indicate your opinion by writing a number from 0 - 10 in the 
space provided. 

1) If you definitely think this student has the ability 
and personal <|ua£ities that she needs to become an 
effective teacher. V/rite 10 in the space provided. 

2) If you definitely think this student docs not have 
the personal qualities and/or ability to be an 
effective teacher, write 0 in the space provided* 

3) Use numbers 1 to 9 to indicate another degree of 
certainty about her effectiveness or ineffectiveness* 
A response of 5 indicates 37 ou are uncertain, or the 
data are inadequate for a judgment. 
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JUDGE miNG bKEEO? (continued) 



The following ecalc way help you keep these directions in mind. 



0 


5 


10 


12 3 


U 6 


7 6 9 


Certain about 


Uncertain 


Certain about 


ineffectiveness 




effectiveness 


Place your estimate in 


this box# 





If your estimate is 5 or below, indicate your reason(s). 



If this student were to teach in a large school system in an 
urban area, would you change your estimate of her success? 
yes ^no 

If yes, place your estimate in this box# 



If this new estimate is 5 or below, indicate your reasonCs)# 




